1. Field of the Invention
The present invention generally relates to computer software, and more particularly to a method of characterizing user interactions with a graphical user interface (GUI) of a program application for later use in automated execution of the GUI, such as in quality assurance testing.
2. Description of the Related Art
Computers and computer software have become ubiquitous in modern society, whether in powerful systems such as servers and technical workstations, or simpler hand-held devices such as smartphones. It is often desirable to automate the execution of a software application for a variety of purposes including testing, development, demonstration, or business process automation. Automated execution of an application is typically carried out using a script prepared by a designer or tester, i.e., a series of commands at the user interface level of the application. Early computer systems and applications used rudimentary command line (text) interfaces and basic script formats, but modern systems provide a more intuitive approach with graphical user interfaces (GUIs) that can be manipulated by other means such as a special keystroke, a keystroke combination, a graphical pointing device (mouse), or a touchscreen.
The advent of GUI-based applications has greatly increased user friendliness and acceptance of software applications but has lead to a concomitant increase in the complexity of the interface. Not only has the interface become populated with a larger number of interface objects and varieties of such objects (e.g., text fields, action buttons, check boxes, radio buttons, hyperlinks, etc.) but there is also a heavier reliance on these features of the GUI. This interface revolution presents increasingly difficult challenges for automated execution of software applications, particularly in the context of quality assurance. While it is important to ensure that any software application is working as intended, it has become necessary to diligently explore the user interface in order to adequately test common features, special functionalities, logic paths, and programmatic aspects of the application.